Why are Analogue Graphics and Natural Language both Needed in HCI?

ion allows natural language to 'directly' represent abstractions over experience in the AG domain. Such abstractions cannot be represented in analogue graphics. A simple example is that it is impossible to graphically represent colour in general. Because of their inherent specificity, analogue graphics have a limited potential for representing abstractions as compared to the corresponding natural language expressions. This is a profound advantage of linguistic expression which seems to reflect the fact that our repertoire of internal representations includes a large number of general and stereotypical concepts in the AG domain in addition to specific mental models. Such concepts can be conceived of as organised into abstraction hierarchies. The colour green, for instance, is already an abstraction which cannot be represented as such in analogue graphics. At a higher level of abstraction, the concept of colour subsumes all our abstract concepts of individual colours. At a still higher level, the concept of visual properties of entities (almost) subsumes the abstract concept of colour together with other concepts. Natural language allows us to freely focus on the appropriate level of abstraction. Whereas colour in general cannot be represented in analogue graphics, the full colour spectrum can be represented in analogue graphics to an arbitrary degree of resolution. Generalising this observation, it would seem that any part of the AG domain can be represented in analogue graphics, to an arbitrary degree of exhaustiveness, namely as collections of specific instances. After all, the AG domain concepts of natural language are built from specific observed instances by the neural circuitry of the brain. However, communication in natural language would be impossible if we always had to include information on such specifics. Instead, natural language makes it possible to navigate freely at the abstraction levels above the specifics in the AG domain to realise particular communicative purposes at the constant price of operating within an interpretational scope. 6.2 Relevance Decidability Given their non-focused character, it can be difficult to decide with respect to a piece of analogue graphics what is and what is not relevant to a specific representational or communicative purpose. It can therefore be difficult or impossible to identify the representational purpose behind a piece of analogue graphics in the first place. Given their focused character, the corresponding natural language expressions do not raise this problem. This is not to deny, of course, the existence of irrelevant discourse. But natural language is 'made for relevance', i.e., for making relevant descriptions at appropriate levels of abstraction. Relevance does not pose a problem for linguistic expression in the sense in which specificity poses a problem for natural language. There is reason to believe that far more cases of communication error arising through the use of natural language arise from lack of specificity than from lack of relevance (cf. Bernsen & Svane 1994). "Be (sufficiently) specific!" is a much more important injunction to include in a practically oriented set of conversational postulates than is the injunction "Be relevant!". 6.3 Beyond the Analogue Media Natural language expressions can represent many types of entity which lie outside not only of the AG domain but outside of the representational potential of external analogue media as a whole, including highly abstract concepts such as 'truth' or 'justice'. Given the notion of abstraction hierarchies of Sect. 6.1 above, even such concepts would seem to have some basis in specific occurrences. However, the properties of those specific occurrences that make them suitable for creating abstractions such as 'truth' or 'justice' cannot be captured in analogue media of representation.